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FOREWORD 


The  last  decade  has  witnessed  tremendous  interest  and 
development  in  artificial  neural  networks.  They  are  beginning 
to  be  accepted  as  powerful  adaptive  tools  for  solving  a  variety 
of  recognition  problems.  Despite  their  dynamic  success,  there 
is  a  perpetual  challenge  presented  by  the  human  brain,  which  can 
handle  tasks  dealing  with  fuzzy  or  uncertain  data.  In  this 
report,  we  demonstrate  that  merging  of  neural  networks  and  fuzzy 
sets,  where  the  former  is  a  powerful  tool  for  performing  massive 
computation  and  the  later  for  coping  with  uncertainty,  provides 
better  tools  for  modeling  the  human  classification  process. 

This  study  was  partially  funded  by  the  Office  of  Naval 
Technology  (ONT).  Kiran  R.  Bhutan!,  currently  with  The  Catholic 
University  of  America,  was  an  ONT  Postdoctoral  Fellow  at  the 
Naval  Surface  Warfare  Center  while  this  study  was  carried  out. 

This  technical  report  was  reviewed  by  Dr.  Kenneth  F.  Caudle, 
Head  of  the  Advanced  Weapons  Division. 
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INTRODUCTION 


Defense  capabilities,  which  are  being  sought  by  the  Navy, 
require  effective  weapon  systems.  Major  improvements  in  autonomous 
real-time  target  recognition  are  critically  needed  to  achieve 
success  with  the  next  generation  of  smart  weapons.  Neural  networks 
have  attracted  tremendous  interest  in  the  last  decade.  They  are 
beginning  to  be  accepted  as  powerful  adaptive  tools  for 
manipulating  and  solving  a  variety  of  problems  related  to  target 
recognition,  speech  recognition,  stock  market  prediction,  etc. 

Artificial  neural  networks  are  modelled  based  on  the 
biological  nervous  system  and  inspired  essentially  by  the 
neurosciences  and  computer  sciences  fields.  These  networks  provide 
a  greater  degree  of  robustness  and  fault  tolerance  than  sequential 
computers.  Thus  the  decision  making  process  of  the  network  is  not 
globally  affected  due  to  a  local  damage  of  few  nodes.  The  human 
brain  on  the  other  hand  is  capable  of  handling  and  performing  aasks 
dealing  with  fuzzy  or  uncertain  data.  Indeed  the  majority  of  us 
think  fuzzy  in  most  situations,  and  much  of  human  reasoning  is 
imprecise  in  nature.  Thus  human  reasoning  is  not  suitable  for 
formalization  within  the  language  of  classical  logic  and 
probability  theory. 

Fuzzy  set  theory  as  pioneered  by  l.  A.  Zadeh^  and 
investigated  further  by  many  researchers^*^  provides  tools  for 
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modeling  uncertainty  and  ambiguity  which  is  so  prevalent  in  human 
interpretation  and  thought  process.  This  promising  field,  which  is 
a  generalization  of  abstract  set  theory,  allows  us  to  assign  a 
degree  of  membership  to  an  object  for  a  class.  This  degree  is  a 
number  between  ”0”  and  "1”,  where  ''O'*  means  no  membership  and  "1" 
means  full  membership.  For  the  classification  task,  the  human 
mind  evaluates  the  perceived  information  by  assigning  memberships 
to  the  various  attributes  in  the  input  space  and  the  stored 
classes. 

Thus  in  order  to  build  intelligent  systems  for  target 
recognition,  we  must  understand  and  represent  mathematically  the 
issue  of  uncertainty  in  human  knowledge.  We  believe  that  the 
merging  of  neural  nets  and  fuzzy  sets,  where  the  former  is  a 
powerful  tool  for  performing  massive  computations  and  the  later  for 
coping  with  uncertainty,  will  provide  better  tools  for  emulating 
human  classification  process. 

This  paper  reports  development  of  a  preliminary  neural  net 
model  which  can  be  viewed  as  a  dynamical  network.  It  is  aimed  to 
incorporate  fuzziness  into  the  network,  thus  resulting  in  improved 
performance  of  the  recognition  process. 
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DESCRIPTION  OF  THE  MODEL 


ARCHITECTURE 

The  proposed  model  is  a  dynamical  neural  net  model  with  the 
number  of  levels,  as  well  as  the  number  of  neurons  in  the  input 
layer  of  each  level,  to  be  determined  by  the  user.  Each  level  of 
the  network  has  an  input  layer,  two  hidden  layers,  and  an  output 
layer  (see  Figure  1).  The  number  of  neurons  in  the  first  hidden 
layer  at  each  level  is  equal  to  the  number  of  neurons  in  the  input 
layer.  This  number  is  equal  to  the  number  of  features  input  at  that 
level.  The  number  of  neurons  in  the  second  hidden  layer  at  each 
level  is  equal  to  the  number  of  targets,  and  the  number  of  neurons 
in  the  output  layer  is  equal  to  the  number  of  target  classes  plus 
1,  an  extra  node  for  no  match. 

Assume  there  are  n  levels  in  the  network  and  k  targets  with  m 
features  of  each  target  input  to  the  system.  Let  ml ,m2 , . . . . ,mn  be 
the  number  of  features  input  to  the  various  levels  l,2,..,n  then 
ml+m2+...+mn  =  m.  We  denote  the  i-th  feature  of  the  j-th  target  by 
f.j,  and  the  tolerance  associated  with  the  i-th  feature  of  the  j-th 
target  by  Q.j.  We  associate  a  vector  (fj,,  f jj,  •  •  • ,  fji^)  representing 
the  i-th  features  of  the  various  targets  with  the  i-th  neuron  in 
the  first  hidden  layer.  We  also  associate  a  vector  (Q,j,  Qjj/*--/ 
Q^j)  representing  the  tolerance  vector  associated  with  the  target 
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j,  with  the  j-th  neuron  in  the  second  hidden  layer  at  any  level, 
where  t  is  the  number  of  features  input  at  that  level. 

Every  level  k  (except  level  1)  in  the  network  receives  input 
from  level  (k-1)  as  well  as  a  new  set  of  features  of  the  target. 
The  justification  of  having  various  levels  in  the  network  rests 
essentially  on  the  fact  that  the  human  mental  processing  scheme 
goes  through  various  levels  of  classification  for  identifying  an 
object.  We  first  look  at  the  strongest  features  of  an  object  and 
try  to  classify  based  on  that  information.  However  if  the  decision 
can't  be  made  at  the  strongest  level  (that  is,  at  level  1  in  the 
proposed  network) ,  we  then  go  to  the  next  set  of  features  and  try 
to  classify  with  the  help  of  additional  information.  This  process 
continues  until  all  the  features  at  our  disposal  have  been 
consumed.  If  the  first  level  of  the  network  can  classify  the 
target,  there  is  no  need  to  proceed  to  the  other  levels.  However 
if  no  classification  is  made  at  the  first  level,  computation  is 
carried  out  sequentially  at  the  next  level  and  so  on  until  the 
classification  decision  is  made. 

From  the  i-th  neuron  in  the  first  hidden  layer  (representing 
the  i-th  features)  to  the  j-th  neuron  in  the  second  hidden  layer 
(representing  the  j-th  target) ,  we  have  a  vector  denoted  by 
Wij  =  (wilj ,  wi2 j ,  ...,  wimj),  where  winj  represents  some  relation 
between  the  i-th  and  the  n-th  feature  of  the  j-th  target  with  the 
i-th  and  the  n-th  feature  of  the  input  vector. 

Let  a  =  (a^,  aj,  •••,  a^^)  be  the  input  vector  consisting  of 


4 


NAVSWC  TR  91-76 


Input  Hidden  Hidden  Output 
layer  layer  1  layer  2  layer 


Level  k 


FIGURE  1.  STRUCTURE  OF  THE  MODEL 
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features  of  unknown  target  which  has  to  be  classified.  The 
computation  at  each  neuron  in  hidden  layer  1  is  carried  out  as 
follows.  Consider  the  neuron  representing  the  first  feature  of  each 
target . 

Compute  I  f,,  -  a,  1  ,  I  f,2  "  1  '  •  •  • »  I  ^ im  “  I  ' 

k  =  min,„.,,j|f,.  -  aj} 


At  each  neuron  in  hidden  layer  1,  the  network  selects  the  class  or 
classes  (k)  which  minimize  the  distance  between  the  stored  feature 
and  the  input  feature.  We  consider  these  classes  winners  at  that 
neuron.  Among  these  winners  we  select  those  targets  which  have  a 
maximum  number  of  wins.  These  are  our  winners  at  hidden  layer  1, 
and  they  are  the  possible  candidates  for  classification  of  the 
ur’^nown  feature  vector.  The  neurons  corresponding  to  these  target 
winners  become  active  in  hidden  layer  2.  Assume  target  1  and 
target  2  are  winners  at  hidden  layer  1,  that  is,  they  have  the 
maximum  number  of  matches  with  the  input  feature  vector. 
Computations  from  hidden  layer  1  to  hidden  layer  2  are  carried  for 
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neurons  representing  these  targets.  Let  =  (c,,,  be 
the  feature  coefficients  of  target  1  which  are  fixed.  The  output 
at  neuron  1  (corresponding  to  target  l)  in  hidden  layer  2  is 
represented  by 


^^11'  ^21 

i  =  1 

where  $  is  some  nonlinear  function  of  the  input  vector  and  the 
weights.  This  vector  =  (P^^,  P2i/---,Pni)  is  then  compared  with 
the  tolerance  at  that  vector  denoted  by  Q,  =  (Q^,,  Q21 ,  .  •  •  ,Qn,)  ,  and 
the  degree  of  match  between  the  unknown  input  vector  a  and  the 
target  class  1  is  obtained  by  the  method  described  in  classifica¬ 
tion. 


TRAINING  PROCESS 

Let  (a^ , aj,  .  .  . , a^)  be  the  features  of  the  first  target.  We  then 
require  that  the  output  at  the  first  neuron  P,  =  (P^i,  P21 ,  •••»  Ppi) 
in  hidden  layer  2,  at  level  1  be  less  than  or  equal  to  the 
tolerance  vector  =  (Qn,  •••/  Qpi)  associated  with  that 

neuron.  Assume  for  some  k  that  P^,  >=  Q^.,.  Then  the  point  P,  lies 
outside  the  hyper-rectangle*  h(Q,)  in  the  n-dimensional  space  with 
the  longest  diagonal  from  the  origin  to  the  point  .  We  say  the 


*If  one  considers  two  features  of  target  1,  then  Q,  =  (Q,i,  Q^i) 
and  so  the  hyper-rectangle  h(Q,)  in  this  case  is  obtained  by 
drawing  a  rectangle  in  the  plane  with  one  vertex  at  the  origin  and 
the  opposite  vertex  at  the  point  (Q,i,  Qei)  • 
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closest  point  to  in  the  hyper-rectangle  is  a  point  =  (T,^,T2i, 
.  . .  ,  )  ,  where 


‘ici 


^ki 

Okl  ^kl^^kl 


Training  requires  that  we  change  the  weights  and  tolerances  in  such 
a  way  that  h(Q^)  expands  and  =  P, ,  that  is,  P^  falls  in  the  hyper¬ 
rectangle  h(Q^)  .  We  interpret  the  training  of  the  network  as 
creating  weak  or  co-weak  isomorphism^  of  itself  until  it  reaches  a 
stable  state,  a  state  when  T,  =  P, . 

CLASSIFICATION  PROCESS 

For  the  purpose  of  classification,  we  apply  fuzzy  reasoning  to 
assign  a  degree  of  meinbership  between  unknown  input  features  and 
the  target  classes. 

Assume  target  class  k  is  the  winner  at  hidden  layer  1  and  let 
(^ik'  ^2k'  *  •  •  '  output  of  iieuron  representing  target 

class  k  in  hidden  layer  2.  Recall  that  at  this  neuron  a  vector 
(Qik'  Qzk'  •••'  Qpk)  associated.  This  vector  represents  tolerance 
the  features  of  target  k  input  at  level  1.  Let 

’^jk  ~  (Qjk  ~  ^Jk)  ^jk 

where  c..  is  the  j-th  feature  coefficient  of  the  k-th  target  and  is 

J  ^ 
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fixed.  We  define  the  degree  of  membership  m^  of  the  input  vector 
and  target  class  k  to  be 

n 

J  =  1 

=  1 

j  =  1 

where  n  is  the  number  of  features  input  at  level  1.  The  above 
formula  of  assigning  class  memberships  considers  to  a  greater 
degree  those  features  whose  weights  are  high.  If  the  j-th  feature 
coefficient  Cj,^  is  high,  then  larger  difference  (Qj,^  -  would 

contribute  to  a  large  number  for  m^  and  thus  to  a  small  degree  of 
membership  m^  in  the  target  class  k.  If  Cj,^  is  low,  then  mj^  is  low 
regardless  of  whether  input  features  match  the  features  of  the  k-th 
target  strongly  or  not.  The  feature  coefficients  indicate  the 
extent  to  which  the  j-th  feature  of  the  k-th  target  is  important 
for  classification  purposes.  The  Cj^  which  are  the  degree  of 
importance  of  the  features  will  be  computed  using  statistical 
methods.  The  winning  target  is  the  one  for  which  the  degree  of 
membership  is  the  highest,  that  is,  target  k  is  the  winner  if 

m.  =  max  (/j?,) 
liiiL 

where  L  is  the  number  of  targets. 
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CONCLUSION 


A  mathematical  model  of  a  multi-level,  multi-layer  neural  net 
was  developed  which  uses  fuzzy  theory  for  classification.  The 
developed  architecture  would  allow  imprecise  or  incomplete 
knowledge  gained  at  one  level  of  the  network  to  be  forwarded  as 
additional  input  to  the  next  level  of  the  network.  Fuzzy  theory 
allows  us  to  interpret  the  information  in  a  more  general  form  for 
developing  methods  in  pattern  clustering  and  recognition.  For  the 
model  under  development,  a  number  of  issues  still  need  to  be 
investigated.  Some  of  these  are; 

1.  Application  of  fuzzy  theory  in  structuring  neural 
networks  for  handling  uncertain  and  imprecise  concepts. 

2.  Mathematical  representation  of  flow  of  information 
dealing  with  ambiguity  from  one  level  to  the  next  level 
of  the  network. 
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